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ABSTRACT 

This paper evaluates Hly used, standardised tests 
of reading within the framework of four interdependent reading 
eomponentsi (1) decoding speed and aeeuraayi (2} aequraay, fluency, 
and flexibility of determining semantic meaning of words; (3) 
sentence comprehensioni and (4) passage dependent ientenae 
oomprehensionp Among the types of standardised reading aohievement 
tests are the group-^administered survey tests, eriterion^ref erenced 
testing systems, testing eomponents of major basal reading series, 
individually-administered diagnostic reading tests, informal reading 
inventories, state assessment programs, and instriunents of the 
National Assessment of Educational Progress* Types of score reports 
range from a single norm^referenced, global score to highly specific 
component scores based on only three or four test items* There is a 
relatively poor match between current theory and experimental 
research on the reading process and existing standardized tests of 
reading. Suggestions are provided for development of new measures of 
reading comprehension that are more consistent with modern 
theoretical and experimental research on the reading process. Eight 
pages of references are included. (LHO) 



* Reproductions supplied by ESRS are the best that can be made ^ 

* from the original dociment. * 

EKLC 



ficorr (if mnmm Nairn. 

ihii «joeum#»f*! fa? pfOGtiifno 

!f» Oiif jufltiHimit. Ihts (liKUiiiitnH 

Hiuftf4<>sfi,ii ntifefj If) Ihif rijjh! 
hni^iiim khiuM rwfiiitt i ihoir 



O 
GO 



Center for Student Testing, Evffluntlcjn 
and Stundardo 

DELIVEHAULE - AUGUST 1986 



RcHearch on Inotructlonal AHsesamtmt i 
TnBtructlonally ReievanC Reading AaBessrnont 



Road/.ng AssessmGntsi Practice and TLc^netical Perspoctlve^ 

Robert La m 
Project Director 

University of IlllnQia at Urbana-Chainpn igti 



U J DEi>AmMlN7Df EpgeATION 

tOViCAllONAl RESOUWCeS INFORMAtfON 
CiNtiRjEmCi 

recflived Hem Ihs ptffsn gr OfgpnizatiQn 
n*ig»risf!ng ft 
O Minsf ehingsi hivt b#8n midf to imprgv^ 



Grant Number 0ERI-C'-86=0003 



Center for the Study of Evaluation 
UCLA Graduate School of Educatiun 
Los Angelas, California 



IerIc 



The project presented, or reported herein^ was performed pursuant to a 
grant from thu Office of Educational Research and Improytnicnt , Department 
of Education (OERI/ED) . However, the opinions expressed herein do not 
necessarily reflect the position or policy of the OERI/ED and no official 
endorsement by the OERI/ED should be inferred. 



Reading Aseesanients Pvsctlcc and Thaoretieal Perspectives 
Robert L, tlnn and Sheila W. Valencia 
Center for Student TcBtlng, Evaluation, and Teating 
Unlvcrraty of UllnolB at Urbana-Chnrapalgn 

No area of student leornlng is more fundoroental to student 
progress throughout school than readlig. Over the last decade ther« 
have been great advances In the scientific understanding of the 
reading process (National Academy of Educa^ n, 1985). These advflnces 
are beginning to have an impact on the de a of Instructional 
mnterlals and approaches to tQachlng reading (Pearson, 1986). To 
date, however, they have had little Impact on testing and there is a 
relatively poor match between theory and experioental research on the 
reading proces;. and standardlaed reading teits. 

This apparent lack of natch between reading research and 
approaches used to assess reading coaprelenslon provides the focus for 
this review. As was noted by Curtis and Glaser (1983), recent 
research on the cognitive processes involved in reading suggests that 
there are four major, Interdependent components of reading 
comprehension. These are (1) decoding speed and accuracy, (2) 
accuracy, fluency, and flexibility of determining the semantic meaning 
of words, (3) passage dependent sentence comprehension, and (4) 
passage comprehension which involves the developBent of a "coherent 
cognitive model of text meaning" (Johnsf-on, 1984, p. 236). 

The four Interdependent reading components identified by Curtis 
and Glaser provide the framework for this review. Existing, widely 
used, standardiged tests are evaluated within this framework and 
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suggeBtlons for the development of new meflBurefi of reading 
coBprehenslon that arr fstent with modern theoretical and 

experimental researr . ng process are provided. 

ISXkSlX g»d U a (i s o 

Clearly, the le of BtandardlEed reading tests 

available for ub. e Alooae all bcHooI dlBtrictB 

adrnlnlster Btandr • achievement testa each year. Included 

among the stando in use are (1) group-admlniatered survey 

tests - either j. cests such as the Gates-MacClnl t la Reading 

Tests (MacGlnltle 1978) or partB of batteries such as the 

Comprohenslve Tet v .asic Sklllfi (CTB/McGraw-Hl 1 1 , 1982)j (2) 
crtcerlon-referenced testing systems, e.g., the PRI Reading Systems 
(CTB/McGraw-Hill. 1980); (3) the tasting components of major baBal 
reading series, e.g., Macmillan's Testing and Management Resource 
Books (Welnsteln. 1983); (4) 1 nd 1 vl dua 1 1 y-a dml ni s tared diagnostic 
reading teats that are Intended to provide mor? detailed Informaion 
about a student's strengths and weaknesses, e.g., the Diagnostic 
Reading Scales (Spache. 1981); (5) Informal -eadlng Inventories, e.g.. 
the Basic Reading Inventory (Johns, 198i)i (6) state assessment 
programs; and (7) tht National Assessment of Educational Progress, 

The types of score reports are as viirlable as the types of testp, 
ranging from a single norm-referenced, global score to highly specific 
component scores that may be based on only three or four test Items, 
The oral reading sections of some testa, for example, provide 
Information on a variety of student errors such as additions, 
substitutions, omissions, mlspronounclatlons, and reversals as well as 
indications of word recognition and comprehension of text. Separate 
scores for Initial, middle, and final constantsi long and short 
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vowels; consonant blende and dJagrnphfl may be among the detflUed 
Bcores that can be obtained. 

Detailed Information on student perforroance on specific skills le 
also commonly provided by "maBtery teats" that have become a atandard 
part of the currlculun mflterlals sold by the publishers of basal 
readers. The lovcl of detail on these curriculum-embedded tests Is 
usually very flna. For example, short testa for specific objectives 
such as "recognize the phoneme-grapheme correflpondenceu of dlpthongs" 
or "divide for syllabication a two-ayllable word with medial consonant 
letters" (Smith & Arnold, 1983) can be found In assessment systems 
accompanying basal readers. 
Theoret leal ggu ndat Ions 

Despite the wide array and widespread use of testa of reading, 
considerable dissatisfaction with the available tests has been 
expressed by the reading reasearch community 'Valencia & Pearson, 
1986). Some of the dissatisfaction Is illustrated by Pearson and 
Herman's (1985) review of the Gates-McKl 1 lop-Horowltz Reading 
Diagnostic Test, Second Edition (1981). 

The Gates-McKlllop-Horowlta test is Intended to provide 
diagnostic InforBatlon on a number of word recognition skills for 
students in grades 1 thorugh 6. Twenty three scores, for 
characteristics snah as omissions, mispronunciations (e.g., wrong 
beginning, wrong middle, accent errors), and recognizing and blending 
eomaon word parts, are provided. Nonsense words, which are 
purportedly made up of two or more frequently used syllables, are used 
on the syllabication subtest. 

As noted by Pearson and Herman (1985), the test follows a 
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traditional view ot reading on coraposed of a a series of subskllls and 
presunies that the eeparate ossoesment of each subsklll will provida 
dlagnostlc0lly useful 1 nf o rraa t i on. It 1b preclflely Lhiu view of 
readings howavar, that raany reading researahers reject. Pearson and 
Herman (1985) express their concern In this regard ae follows: 

"To asscas sheer knowledge of these 'plecea' outside the conte%t 
In which the student Is eKpeeted to apply that knowledge, l*e., 
whan the student Is really reading real text, gives a biased 
estimate of ability to use that knowledge* Not only does 
assessment devoid of eontext prevent the student from using the 
rich range of resources available in most real text, it also 
prevents the examiner from getting a picture of how the Btudent 
la or Is not able to marshall resources, skills, and strataglas 
to solve the problem of what the text means" (p, 602), 
Similar ooncerns have been expressed by other reading researchers 
about the emphasis on a host of discrete skills In many criterion-^ 
raferenced reading test systems. In h^r review of the PRl Reading 
Systems (CTB/McGraw--Hl 1 1 , 1980), for axample, Hlabert (1985) concludad 
that the "assessment program suffers from the major deficiencies that 
have typically characterized criterion-referenced programs p.* The 
most fundamental Issues involve the assumptions that mastery of these 
objectives constitutes raading and that these objectives are acquired 
in this particular order" (p. 1201)^ 

The emphasis on discrete skills in inconsistent with the emerging 
concaption of raading as an integrative process (National Academy of 
Education, 1985| Stanovich, 1980), So too are the emphases found in 
most individual ly^adminls tared diagnostic reading tests. Indeed, 
there is little thaoretical rationale to support these tests, and even 
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less In the way of evidence that the putatively diagnostic Inforfflatlon 
that Is provided leads to valid Inatuctlonal decisions. 

According to the National Academy of Education's Commlsalon on 
Reading (1985), "[rjeadlng la the proceBs of conBtructlng ffleanlng from 
written texts. It Is a compleK skill requiring the coordination of a 
number of interrelated sources of infornatlun" (p. 7), Skills such as 
the ability to decode words and to assess the eemantic meaning of a 
word are certainly neceBSary for the skilled reader, but reading 
coaprehension requires more than the application of n series of 
discrete skills. Reading is an active process that rsquires the 
integration of information provided by the text with the reader's 
prior knowledge (e.g., Bransford & Johnson, 1972; Johnston, 1983; 
Pearson t, Spiro, 1980). Purposive reading requires se 1 f -monl t o rl ng 
and self-regulatory skills that have come to be known as 
"metacognltlve abllites (e.g.. Brown, 1978| Flavell, 1976). 

Higher level integrative and metacognltlve abilities receive 
relatively Uttle att^stlon on current tests which emphasise dlBcrete 
skills. Instead, teats often emphasiEe literal comprhension rather 
than Inference and rarely go beyond asking a student to find the main 
Idea of a brief passage (Valencia 4 Pearson, 1986). As noted by 
Curtis and Glaser (1983), both reading instruction and achievement 
testing, although adequately addressing lower level reading 
requirements, may be doing so at the expense of higher levels" (p. 
133), They go on to argue that "an alternative view of reading 
instruction and testing Is needed in which (a) the skills involved in 
reading are assumed to be more Interrelated than In a stage model, and 
(b) revisions are made In a way that teaching and testing take into 
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account InteraeClona among components of reading" (Curtis S Glaaor, 
1983, p. 133). 

Recent research In cognitive psychology (see, for example. Curtl« 
& Glaser, 1983; JohnBton, 1983- Stanovlch, 1980) provldeB a 
theoretical foundation for the evaluation of current reading tests and 
suggeBtlons of new approachea to the meflBurement or reading 
comprehenalon. Curtla and Glaser (1983) clearly Illustrated how 
theory and research on cognitive processes in reading can be used to 
apeclfy desirable cha rac t er 1 a t lea of teats of reading coaprehenalon. 
As was noted above, their analysis led to the Identification of four 
major. Interrelated components of reading corap re hens 1 on i (1) decoding 
speed and accuracy. (2) accuracy, fluency, and flexibiUty of semantic 
meaning of words, (3) passage dependent sentence comprehension, and 
(4) passage conprehenslon. 

Although each of these components Is Important for skilled 
reading and may measured separately, the key aspect of this 
framework Is the emphasis on the integration of the components. 
Perfettl and Leagold (1979) Illustrated the essential Interdependence 
of the components of reading by analogy to the components of high 
fidelity systems. "If any component of the system is defective, sound 
quality suffers. The components can be Independently tested and, more 
importantly. Independently Improved. However, Improvement of one 
component may not Immediately Improve sound - Mality (but It may 
Increase the potential of the system to benefit from later 
Improvements In other components)" (Perfettl & Lesgold, 1979, p. 58). 
In an analogous fashion, it is argued that the components of reading 
comprehension must be Integrated to produce meaning from text. 

Perfettl and Lesgold's hi-fi analogy forcefully illustrates the 
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key role thut reading theorists attach to the integrntlve aepects of 
reading, Ao 1« true of most analoglofl, however, It con be carried t 
far. There Ia evidence, for ^Kample. that the components of rending 
unlike those of a hl-fl Bysten. are not only Interactive but are 
compensatory. Stanovlch (1980) has reviewed a large body of 
theoretical and experimental work that suggeBte the need for what he 
has referred to aa "an integratl ve-coapenaatory model of Individual 
differences In reading ability" (p. 63). 

The need for « compensatory model la best Illustrated by the 
reliance on context to Identify words. Skilled readora apparently 
rely little on context to Identify words (McConkie & Zola, 1981) • 
because their word recognition and decoding alUlls are so efficient 
that there is seldom need to depend on the context to identify 
individual words. Top down models of reading that hold that readers 
are continually hypothesis testing as they read are laplauslble 
because they require that readers are able form and test hypotheses 
"based on cooplex syntactic and seoantlc analyses ... In less then thi 
few hundred milliseeonds that is required for a fluent reader to 
recognlae most words" (Stanovlch, 1980, p. 34). if anything, the 
evidence suggests that less-skilled readers rely more on context for 
word identification that do more-skilled readers (see Stanovlch, 1980. 
for a review of several studies supporting this conclusion). 

The use of context by readers with poor decoding skills 
illustrates the compensatory nature of reading. It is not that good 
readers lack the ability to use context. Indeed, there Is evidence 
that good readers tend to be able to predict words that are missing 
Iron a passage better than poor readers. However, it does not 



necoBsarlly follow that because gnod readero have euperior predictive 
abilities that they rely on these oblltlen to recognize worda 
(Stanovlch, 1980). Rapid word Idcnt If cation by okllled readers occurs 
automatically and frees cognitive rcBources for higher level 
processing and interpretation of the text (Curtis & Glafler, 1983). it 
seeme likely that it la giaply more efficient for readara "with well 
developed decoding skills to directly identify words in a teflt-drlven 
Banner than to *predlct^ words bneed on context" (Juel, 1980, p. 49^. 

Readers with less ability to reeognlEe Individual words 
automatically may compensate for this deficit by placing greater 
reliance on context and stored knowledge. Such compensation Is only 
partially effective, however, because It cooea at a cost. It requlrtc 
the use of cognitive reaourcee that are needed for the higher-level 
processing that is essential for the construction of meaning from the 
tejtt. Thus, as in Perfettl and Lesgold's analogy to a hi-fi system, a 
weak cofflponent can llait the overall quality of the system because the 
other coBponents can't work to capacity, much less, fully compensate 
for the weak one. 

The four interrelated components of reading Identified by Curtli 
and Glaser (1983) provided a framework for reviewing currently used 
tests In the following sections of this report. Given the large 
number of published teats and the aubatantlal degree of similarity 
among many of the tests that are designed to serve a given purpose, an 
exaustlve review of all published tests was not attempted. Such a 
review would be highly redundant. Hence, major examples that 
illustrate the range of approaches In each area were reviewed. For 
each component, an attempt was made to analyze the tests in terms of 
recent research on reading and to use that ressarch as a basis for 
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suggesting future dlroctlons for the testing of these cowpon-nir, 
PfeCQdi ng Speed litid A ecu raey 

Ae has already been suggested, word decoding is n key coffiponcnt 
of the reading process - one of several Interdependent processes that 
contributes to ikllled reading. Tho work of Samuels and LaBerge 
(1974) on automatic processing of words has suggested that autonsatlc 
decoding of words la a key to the reduction of attentlonal deaanda of 
the reading task. When these defflands of word tagka are reduced, more 
attention Is freed to be allocated to otheri and perhaps higher level, 
procenses such nn € one t rue 1 rig a coherent model of the text, 

Aa ftummarlEed by Stanovleh (1980), speed of word recognition 1© 
Important because rapid rrocesslng of Information In short term memory 
enables the reader to utilise higher level Integrative comprehenfl Ion 
procasses that operate on the Information stored there (Lesgold £i 
Perfettl, 1977| Parfettl a Lesgold, 1977). Numerous studies have 
suggested that rapid, context-free word identification is a major 
factor distinguishing skilled readers from less skilled readers and 
that speed of word recognition continues to increase after the 
leveling off of automaticlty at the second or third grade levels 
(Curtis, 1980; Parfetti & Hogaboaa, 197S| West & Stanovich, 1979j 
Guttentag & Halth, 1978| Hogaboam 4 Perfetti, 1978), 

The available evidence strongly supports the conclusion that the 
ability to identify words accurately and quickly is critical to 
becoming a skilled reader* Although some readere are able to 
recognize words adequately, they respond so slowly that meaning is 
often lost. At the other eKtreme, there are readers who will "read" 
words vary quickly but may fail to get any meaning from them because 
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th« word* are Incorrectly Identified. In fact, Btflgpa ot flight word 
acquisition (Adams & HlgginB, i985; Sarouola & LaBurg,^, 1976; HcCortniek 
& DamuelB, 1979) auggoot that atudenta flrot muBt develop nccuracy 
with word idcfntlf Icntlon before they become autoraotie or Incroasc 
Speed of identification. While Boine Btudles have found Blgnlflcance 
when examining reading ability using only acciirflcy of word 
Identification (AdamB & HugglnB, 1985; Juel 4 Roper-Schneider, 1985), 
others have focuBod» and found Blgnlflcance when looking at speed of 
word procEBslng alone (Blenlller, 1977-78; Hogflboam & Perfettl, 1978; 
Perfettl, Pinger & Hogaboam, 1975). 

Some reaearchers, however, have Invastlgatad good and poor 
readers in termB of Bpeed and accuracy of word Identification. For 
©xanpie, Shankweller & Llborman (1972) indicated that word Hat 
accuracy for students In gradeg 2 to 4 waa highly correlated with 
reading performance on paragraphs. However, after ejcamlnlng the 
correlations between latency, accuracy and paragraph reading for one 
group of third graders, they concluded that slow rate of individual 
word Identification say contribute as ouch aa inaccuracy to poor 
perfornance on paragraph reading. 

There are numerous individually-administered tests that asaess 
the recognition of Individual words. The word recognition section of 
the Wide Range Achievement Test (Jastik & Jastlk, 1978) is 
illustrative of the general approach that Is often used. Listi of 
letters and words ate presented to children and the accuracy of 
pronunciation is recorded. On Level I of the WRAT, which Is Intended 
for use with children aged 5 through 11, the words are arranged 
H approximately in order of increasing difficulty and testing is 
dlcontinued after 10 coosecutlve errors. 

ill: 10 
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The word r«cognltlon section of the WRAT 1« onsy to ndinlnleter 
«nd yields flcorco with relatively high ftillabUlty. It doca not 
provide a mcaaure of «p«d of word rcicognitlon, howcvor. Furthermore, 
the accuracy ocoreB ara 1 n t e r pre t a b 1 e only In norra-ref erencad termB. 
This la BO because there Ib no sound rationale or syBtematlc basis for 
the selection of words for the test. 

Several informal reading Invantorles Include word racoBnltlon 
flections. The Bflslc Reading Inventory, Second Edition (Johns, 1981), 
for example, includes a "word recognition in isolation" subtest. The 
word llBt la said to be graded and nay be administered as a timed or 
untioed test. Although it has been suggested that the comparison of 
timed and untlmed perforaianue may by useful (PlesBas, 1985), speed of 
vocaUzatlon is not directly aBsosBed. The basis for the eel«ction of 
words is unclear and, as Is typical of several informal reading 
Inventories, no statistical support of the reliability or validity of 
the test is provided. Nor Is the evidence provided that would justify 
the use of the results on the word recognition section to determine 
initial placement in the test's reading paBsages. 

The Ekwall Reading Inventory (Ekwall, 1979). a similar Informal 
reading inventory, uses a graded word list for determining initial 
placement into reading passages. As was true of the Basic Reading 
Inventory, there la a lack of technical information or justification 
of the choice of words for the word recognition portion of the 
Inventory. The intended use of the word recognition in isolation 
sections of tests such as the Ekwall and Basic reading inventories is 
relatively limited, however. 

Some published teste focus exclusively on word recognition. Two 
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such te«ti are the Inntant Words Criterion Teat (Fry, 1980) and the 
Po|»« Inventory of haalc Reading Skills (Pope, i97-'0. The Inatflnt Word 
Criterion Test cdnslBts of 300 common words, which the author claltDH 
^'^^ BMl f-omman words In English. Aa noted by Elkliitt (1985, p. 

692), "Injo roferencett are given to support thin claim, but an 
Inapectlon BuggeBts that theoe wordfl arc Indcod bapic sight 
vocabulary." It le presumed that theso words should be recognlised 
Infltantly. Children are required to read the llBt of words and the 
examiner records words that are "mlsBed". Speed of recognition is not 
assessed and the criteria for recording a failure are not specified. 
"Neither Is there any indication given of how mucn time should be 
allowed to the pupil to answer each Item, which Is auroly important in 
view of the word 'instant' in the title of the teat and the ItBplied 
aBsumpclon" (Carroll, 1985). It la fllmply asauiaed that words that are 
Blssed should be taught since recognition of these words is needed for 
reading. Thus, only Item scores are suggested. There is no 
supporting evidence of reliability (either agreement aoong examiners 
or test-retest) or of validity. 

The Pope Inventory of Basic Reading Skllla is an individually 
administered teBt that le intended to assess word attack skills. 
Section II of the Inventory is used to assess a child's basic eight 
vocabulary by presenting the child with a deck of cardB with one 
coBfflon word printed on each card. The child is Instructed to first 
sore the cards into two plleai known wordo and unknown words. The 
child Is then asked to read aloud the "known" words and the number 
read correctly Is used as an estlnate of the child's sight vocabulary. 
No noras, reliability, or validity data are provided. 

The Blemlller Test of Reading Processes (Biemlller, 1981) is one 
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of the few published tests that attetnptu tomt-rtnure epced of word 
rccognlclon dlriiccly. Speed In naming letters, worda In Iflolatlon, 
and words In context le mecBured. The words In Isolntien part of the 
test Ifl divided into two llsta of 50 words each. The flrat list is 
drawn from pr tnjc r- 1 e v e 1 vocabulary and the second from middle- 
elementary school level vocabulary. Percentile scoreB (90th. 75th, 
50th, 25th, or 10th) ar« reporti.d. The percentiles, however, are 
based on a sanple of 340 children from 17 Canadian flchools, and are 
not representative of any clearly defined population. 

The precise basle for iafflpling wordB to be Included on the lists 
Is unclear. Although the dlreetlons for admlnlBtratlon are quite 
specific, it ig likely that considerable practice would be needed to 
obtain accurate measures of speed. The following cooments by Pry 
(1985) are worthy of note in this regard. 

"The problenft of accurately recording reading tine ... indicate 
the examiner would need some training and experience. It ig 
Important to keep the child reading. For eitample, the author 
stfltes, 'If the child pauses over a word or rereada a line, stop 
the stopwatch until the child is reading words she/he has not 
read before.' In addition, 'If there are more than three delay- 
type errors (as opposed to misreadlngs without stopping), the 
data will be invalid, and testing should be stoiped.' This is a 
lot for a novice to remember when aeconds count" Fry, 1985, p. 
194). 

Although not really a test, the lOX Basic Skills Word List (lOX 
Assessemnt Associates, 1980) deserves mention as the final instrument 
In this section. As the name suggests, the lOX Basic Skills Word List 
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Is « 1 18 t of word 8. A total of 7 , 3 t 8 words « re Inc 1 udcd in the 
which iu orgalzed by grnda level (frota I to 1?). tha wordfj were 
Belfcted uBlng Infornmtion from noveral aourccs using the following 
crlterU: "(1) the froqucncy with wllch words occur In roadlng 
textbooks, (2) the froquoncy of words In generally read materialB. and 
(3) children's demonstrated ffloHinrlfcy with particular words" (lOX 
Asses^fscut Associates, 1980), 

The Initial list and grade lavel aflBlgnmentis were taken frotn 
Taylor, Frnckenpohl, White, Nleroroda, Browning, and Blrfner's (1979) 
core vocabulary list. Modifications of grade level assignments and 
additions to the list were based on faBlllBrlty of words to children 
at different grades ob indicated by Dale and O'Rourke (1976) and 
Sakley and Fry's (1979) 3000 nost frequent words from rhe Carroll, 
Davles, and Rlchoan's (1971) Word Frequency Book . 

Though not a test in the usual sense. It is suggested that the lOX 
Basic Skllli Word List might be used to construct InforBal teats of word 
recognition for use in "diagnoi is". The list is also Intended as a 
resource for judging the readability level of texts or passages to be 
used on tests and for identifying words for "direct teaching". The 
rationale for the suggested diagnostic use in unclear, however. More 
Importantly, the notion that words should be identified in this manner 
for direct teaching is not Justified. The heavy emphasis on 
individual words at the expense of time devoted to reading complete 
stories or text is at best debatable (see, for example, Anderson 4 
Freebody, 1983). 

This brief review of word recognition measures leads to two 
generalleatlonsi (1) speed of word recognition is only rarely measured 
directly and (2) the selection of words for tests is often more of an 
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art Khnn a aclencc. there 1b seldom a claar justification for the 
irtclualon of particular wordfl on d teec. But the eolcctlon of words 
for tests of speed and accuracy of word Identification la certainly 
crlcltal to meaningful interpretation of results, in other than a 
norm-referenced sense, and to relevant clasaroom Instruction. Our 
review of the reacarch suggests a number of factorB that are 
potentially relevant In the c laBti 1 f lea t Ion of words, including word 
frequency, IncluBlon and OfflphaBls In the curriculuB, the approach of 
the Inetructional program, orthographic complexity, word type, and 
word length. Based on our review, however, we have concluded that the 
three factorfi that are most critical for creating apeclf ications for a 
test of speed and accuracy of word Identification are (1) word 
frequency, (2) expoBure to words in the curriculum, and (3) the 
instuctlonal program. 

Skilled readers are faster than leSB-silled readers at 
identifying high frequency words (Biemiller, 1977-78; Curtis, 1980| 
Perfettl, Finger & Hogaboaa, 1978| Perfetti & Hogaboam, 1978; West 4 
Staovlch, 1979). At the same tine, however, Perfetti & Hoagaboam 
(1975) have demonstrated that differeaces between good readers and 
poor readers are even greater for low frequency and pseudowords. This 
suggests that speed is not just familiarity with particular words. In 
fact Juel & Roper-Schneider (1985) found that first grade students 
with a more Intensive phonics Instructional program were more 
successful at correctly pronouncing words not seen before than 
students in a more vieual word recognition program. 

West and Stanovich (1979) have cautioned that one of the reasons 
researchero may have been unable to demonstrate de velopnental trends 
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m automotlclty ia that many of the words uaed on monBurcs have been 
coo eiiay. That Is, there is u sort of celling effect. Thin aame 
Ciiutlon Is applicable In the construction of lists of words for ceata, 

Related to the loaue of frequency, 1b expofiura to worda or the 
currlcular validity of those words. Juel & Rppcr-Schneldor (1985), 
for example, found that the typei of wordi In first grade texts oxert 
« more powerful Influcnco on children's word recognition strategies 
than the method of instruction. AddltlonaUy, they found that 
repeated exposures to worda, the number of repetitions in the basal, 
was a elgnlflcant factor in accuracy of basal word Identification. 
Both these flndingB euggest that incraaBed reading nay provide added 
expoBure to words that may In turn increase word Identification speed 
and accuracy. These findings also suggest that the match between the 
words en a test and the textbooks and instructional program Baterlals 
may be Important determinates of the instructional validity of a test. 
Semantic Meaping 

Although the research reviewed above makes a strong ease for the 
iBportanee of being able to recognize words aecurately and quickly, 
this ability represents only one of the interrelated components needed 
by a skilled reader. The breadth and depth of an indivdual's 
vocabulary knowledge Is also vital and provides a good predictor of 
reading eoBprehension ability (Thorndike, 1973). it is almost self 
evident that a reader who does not know the meaning of a large number 
of words la a passage will have great difficulty comprehending it. 
There is considerable evidence to support this common sense conclusion 
(see. for example, Anderson 4 Freebody, 1981), However, as Curtia and 
Glaser (1983) have cautioned, simply learning word meanings "does not 
always lead to conpreheoslon laprovement (Jenkins, Pany, & Schreck, 



, I9?8; Tuln.«„ , Brady. 1974). P.r vocabulary InatrucUon Co hav. «„ 
effect on reading comprc^henslon. It needs, at a mlnlmu., to go beyond 
dictionary deflnUlonB to doop und.rs tand ingo and related ldoa« 
(Draper & Moeller, 1971), 

Anderson and Freebody (1983) have „ade a convincing case that 
estimates of vocabulary sl.e are potentially important both for 
purposes of policy decl«lono and for purposes of asa„Bl„g Individual 
dlfferencas. As they deBonstrata. however, there are order=-of- 
«agnitude differences In the existing estlmatee of vocabulary sl.e. 
Anderaon and Freebody cite two reaeona for the widely divergent 
estimates. First, th.ro ar« differences in the definitions of the 

doaalD of potential words < « «u„#. „ 

H emiQA words, I.e., what counts as a aeparate word and 

bow inclusive the word list Is for which eatiBates are sought. 
Second, the estimates depend on the methods used to measure knowledge 
of selactad words, 

Rtadlng testa commonly report separate scores for vocabulary 
knowledge, but they do not purport to give an estimate of a teat 
t.ker'. vocabulary si.e. Rather, intrepretatlons are generally baaed 
on normative comparisons. Hence, before considering possibilities for 
obtaining estimates of an individual's vocabulary size we will review 
.o«e of the major measurement approaches used on staodardi.ed tests of 
vocabulary knowledga, 

A variety of formats are used on teats of vocabulary knowledge. 
By far the most common procedure, however, is to use multiple choice 
questions. Either isolated words or words in cont^t are commonly 
presented and the test taker Is instructed to select one of four or 
five options that has the same or most similar meaning. When words in 
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context ore preaented, the context may conslBt only of a two or three 
word phrase or ono or more complete sentences depending on the test. 
At higher eradee antonyms, rather than synonymn arc alBo frequently 
used . 

Given the high degree of elmllarlty among teflts of vocabulary 
knowledge used In the elementary grades, only a few tests will be 
described here. The Gatea-MacCinltie Reading Tests (MacGinitle, 1978) 
Is a well known and widely used reading test series. It consists of 
seven levels that together span grades 1 through 12. As is typical of 
such series or of comprehensive achievement test batteries, vocabulary 
as well as comprehension is assesBed at each level. In the lower 
grades, vocabulary is assessed by having test takers select one of 
four words that describe a picture. Starting at grade three, test 
takers are required to select a word or phrase that has the most 
similar meaning to the word preBented in the item stem. Using a 
variety of sources, "[w]ords were selected to characterize those 
likely to be found in reading materials in the grade range covered by 
each test level" (Rupley. 1985, p. 595). The teat has norms both for 
in-level and out-of-level testing and high internal-constls tency 
reliabilities (.90 to .95). 

The Word Meaning test of the Nelson Reading Skills Tests, Forms 3 
and 4 (Hanna, Schell, i Schrelner, 1977) uses two types of Items, both 
of which require the test taker to select the nearest synonym from 
four options. The stem consists of either Isolated words or words in 
phrases. The vocabulary for the tests was selected from humanities, 
social studie- and science texts. Norm-ref ereneed scores with good 
split-half reliabilities (.89 to .93) are reported. 

The vocabulary section of Level E of the Comprehensive Tests of 
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Basic Skills, Forms U and V, (CTB/McGraw-Hi 1 1 , 1982), which Is 
typically used in Che third grade, uses three item formats and 
Illustrates 8om« of the range of items types found on widely used 
teats of vocabulary knowledge. The first item type presenta a two or 
three word phase with one underlined word (e.g., "very large") aa the 
stem and the test taker is told to select the "word below that means 
the same or about the aame as the underlined word". For the above 
sample item, for example, the the optiona are "big", "old", "brave", 
and "strong". The stem for items In the second section presents two 
underlined phrases (e.g., "a bflseball player and something used for 
£ourln£") and the child Is Instructed to pick the word that fits both 
("glass", "bottle", "catcher", or "pitcher"). The third item type has 
one or more sentences in the stem in which one word is underlined 
(e.g., "Billy sat crying on the steps. His dog was lost. He was so 
Higerable."). The child is Instructed to select one of the four 
optional words that means the same or nearly the same as the 
underlined word ("cold", "sleepy", "excited", "unhappy"). 

These ItemB Illustrate that tests of vocabulary may tap a variety 
of kinds of knowledge about a word. As noted by Curtis and Glaser 
(1983, p. 137), "vocabulary items differ in the extent to which they 
assess individual's abilities toi (a) recognize a correct meaning of a 
word; (b) determine which of several correct meanings Is appropriate 
In a particular contexts and, in some cases, (e) figure out an unknown 
word's meaning from context." A given child might use all three of 
these strategies in answering the three sample items quoted above from 
the CTBS. Clearly, It could not be safely assumed that a correct 
answer for the word "miserable" Implies that the child would also have 
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answered In the fiflmo wny If the word wore presenti^d in l,ioUUon. The 
word miserable is a low frequency word in third grade materials, 
ocurrlng only twice in the over 800,000 words from third grade texts 
analyzed by Carroll, Davles, and Rlchnan (1971). Yet with the 
supporting context it was Judged to be easy enough to use as a sample 
1 tern. 

Anderson and Freebody (1983) report a number of fairly wide 
dlacrepencies between the percentage of children who claim to know a 
word and the number who give the correct answer to a multiple choice 
Item Involving that word. The dlecrepency can go in either direction, 
depending on the nature of the multiple choice item. For example, 
only 19% of their sample of fifth grade students said that they knew 
the word "judicious", yet 511 of the sample gave the correct answer to 
the following multiple-choice itemi 

"A Judicious decision is made - 

1. quickly 3. foolishly 

2. wisely 4. cleverly" 

On the other hand, 96% of the sample said they knew the word "manage", 
while only 28% gave the correct answer to the following multiple- 
choice Itemi 

"If you manage on your allowance, you - 

1. spend it 3. get along 

2, save It 4. waste It" 

Anderson and Freebody found a closer correspondence between 
interviews of students about their knowledge of word meanings and a 
simple yes or no test of whether or not a student knew a word than 
with the results of multiple choice items. They concluded "that when 
the word tested In a standardized nultlple choice Item is difficult 



something about the Item will tend to give away the correct answer, 
whereas when an easy word Is tested the Item will tend to load the 
student away f ron the correct anawer" (Anderson & Frcebody, 1983, p. 
238). ThlB tendency Is likely to be the conBequence of reliance on 
Item analytla statlatlca In the Belectlon of teat ItemB. On norn- 
referenced testi, preference is traditionally given to Itema of middle 
difficult. Extremely easy and extremely difficult items are excluded. 
Thus, for an easy word to be Included fine dlBtlnctions may be 
required for the item to pasfl the item analysla screen, whereas the 
converse raay be true for difficult wordB. 

Indirect support for Anderson and Froebody's conclusion Is 
provided by a comparison of item difficulties (as Indicated by the 
location parameter from the three parameter logistic modal) and the 
frequency of occurrence of the target words. In general one would 
expect a substantial relationship between the frequency of occurrence 
of words in books and knowledge of word meanlngB. However, for the 30 
target wordB in the vocabulary section of Level E of the CTBS Form U, 
the correlation between the Standard Frequency Index found In Carroll. 
Davles, and Richman (1971) and Item difficulty (location parameter) Is 
only -.09. Thus, there Is only a slight tendency, if any, for the low 
frequency target words to be more difficult than the high frequency 
words. This poor relationship is likely to be attributable to the 
requirement of making finer distinctions for high frequency words 
(e.g., "famous" and "whole") than low frequency words (e.g., "chuckle" 
and "thaws"). More clues are also apt to be given by the context 
accompanying low frequency words. 

For norm-referenced interpretations or for purposes of 
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prediction, the ambiguity of the information provided by a multiple 
choice item about a ntudent'o knowledge of the n*.flntng(8) of a word is 
unimportant. However, if the goal is to eBtimate vocabulary alzo or 
to have an Indication of whether or not Btudenta know the mennlngs of 
words found in Instructional materials, thiB ambiguity is undeBlrable. 

An alternative approach to meaBurlng word knowledge has recently 
been investigated by Anderson and Freebody (1983). They used a simple 
yes/no method where a student is required only to Indicate the words 
he or she knows by meanfi of a check or some other device (e.g., 
pressing a button). Thla approach ellmlnateB extraneoua factors such 
as those due to conteKt or the nature of the multiple choice options 
that are selected. The obvious drawback ig that students may differ 
in their propensity to respond "yes" to words that they don't know or 
to respond "no" to words about which they are unsure. 

To guard against these poss ibl 1 Itl es , Anderson and Freebody 
intersperse pseudowords with the real words on the test. The 
proportion of words known is then estimated by the difference In the 
proportion of yes's to words (hits) and the proportion of yes's to 
pseudowords (false alarnB) all divided by one minus the proportloQ of 
false alarms. A large number of words and pseudowords can be 
presented In a fairly short period of time using this techlnlque. 
While not without difficulties (e.g., the problem of specifying the 
rules for sampling words and for constructing pseudowords), the method 
has considerable appeal for some purposes. 

Anderson and Freebody's yes/no method aay prove useful for 
obtaining criterion-referenced estimates of some, but not all, aspects 
of a student's word knowledge. Curtis and Glaser (1983), for example, 
have suggested that there are three aspects of semantic word meaning 
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that may be Imporf.ant to distinguish for purpoeca of measurement, 
these are accuracy, ne^lblllLy, and fluency. Accuracy "reflects 
whether or not there la my appropriate eemanrlc knowledge lis oemory 
associated with a word" (Curtla * Claaer, 1983, p. i3&). Flexibility 
refers to the depth of an Individual's knowledge of word weaning and 
fluency refers to the speed of acceos to the neanlng of a word, Ae in 
the case of slow decoding, slow acceaB to the neanlng of a word uaes 
cognitive resources that arc nueded for processing sentences and 
cois^prehendlng longer sognents of text. Selecting the most appropriate 
of several meanlngB of a word is cBsentlaJ to comprehe na Ion of the 
meaning of a sentence ao a whole and the adequacy of the overall 
comprehension of the meaning of the text can depend on the rlchneas or 
depth of knowledge about particular words. 

Current published tests of word meaning do not distinguish among 
the aspects Identified by Curtis and Glaser. The lack of such 
dlBtinctlons is of no real concern If the goal of measurement Is to 
rank order students or to predict future performance, but the 
distinctions could prove important for purposes of planning 
Instruction. Anderson and Freebody's yes/no procedure aay provide a 
useful alternative for assessing accuracy and possibly fluency. 
Other, fflore adaptive techniques, are likely to be needed for obtaining 
measures of flexibility, however. 
Sentence Comprehension 

The third of the four components of reading suggested by Curtis 
and Glaaer (1983) was called sentence processing. Sentence processing 
refers to the "integration of each incoming sentence Into the memory 
Structure that exists for what has already been read" (Curtl* & 
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Clitser, 1983. p. 138). S«eh l.Ut^ijr^tlQn may Involve a nlmpU matching 
of words or concopt« in t>n«? a«nt«ncc with thu»« In the Ifflmedlately 
preceding eentence. For cKawplc, a pronoun may b« macchod with « noun 
in the prevlouB Hontence. Now Informatlnn contnlned by the second 
sentenco may provide an elflboration of a concept in the firat 
sentence. Comprehension in this cane requlrca an Integration ol the 
new Inforoatlon with the concept in Bhore-term memory. 

Integration involving eimple Batching of words or the linking of 
new Infornatlon to elaborate a previoua concept in Bhort-tero Boaory 
can be illuBtrated by the first two sentences of the first reading 
passage of the Elementary Level (arades 3.5-4.9) of the Meteropol Itan 
Achievement Testa. Fifth Edition. Form JS (Pr«acott, Balow, Hogan, & 
Farr, 1978). 

"Our block used to have many oak trees along the street. 
It was quite shady and cool in the Bunmer," 
Tha second aantence is linked to the first by the pronoun "it" and the 
concepe of the oak trees Is elaborated by "shady and cool". 

Of course, the Information In a new sentence does not always 
Involve such direct matches to Information contained In short-term 
memory. When it doesn't, the reader must rely on other Integrative 
processes. This may involve searching long-term memory for 
tnformatlon encountered earlier in the passage or the prior knowledge 
the reader brings to the task. Curtis and Glaser (1983) used the 
following four sentences to Illustrate the need to reactivate 
Information stored in long-cerB meBory. 

"A thick cloud of smoke hung over the forest. Glancing to the 
aide, Mary could see a bee flying around the back seat. She 
pulled off the road and rolled down her window. The forest was 
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on fire" (pp. 13B~139). 
Beenufle the aecond and thtrd eantenceo dcol with a topic different 
than the first, the "cloud of omokc" Ib no longer llk«ly to be in 
Bhort term memory. Ren e t 1 vn t ion o( thla Inforootlon la Blgnolffld by 
the retntroductlon of the word "foroGt" (CurtlB 4 GlaBer, 1903). SucI 
Integration of Informaclon acroiiB aontencoB Is critical to a good 
understanding of the text. 

Lacking prerequisite prior knowledge or having an Inadequata 
memory of the previous Information supplied by the pasflage reduces 
comprehension. "Both the efflcloncy of decoding Bklllfl and the 
Bufflciency of semantic word knowledga can affect the oa nipu 1 abi 1 1 1 y 
and availability of paBsage Information. Indlvldiiali «ho have auch 
reading difficulties tend to be lesB able to hold apeclflc words from 
a passage in short-term memory (lerfettl & Goldman, 1976)" (Curtis & 
Glaser, 1983, p. 139). 

Sentence comprehension is frequently asfleBaed In standardized 
reading tests by presenting questions that paraphase parts of the 
text. A large proportion of questions on some tests can be answered 
by direet comparison of the question to a sentence In the text without 
oecessarily being able to understand the sentence (Anderson, 1972; 
Curtis & Glaser, 1983). It ha. also been deaon.trated (e.g., Tuinman, 
1974) that a substantial fraction of questions on some standardized 
reading tests can be answered correctly without reading the text based 
simply on prior knowledge. To assess Integration, It is Important 
that questions (1) be passage dependent, i.e., not be answerable on 
the basis of prior knowledge without reading the target sentences and 
(2) require understanding rather than being answerable on the basis of 
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word Watching. 

The better Btondardl^ed tcBtw hnve attempted to addrwHH thc8« 
concerns. Qu««tlon8 that require ItUCKr/itlon of Inf ormatlon from 
different nentenceo and that have a low likelihood of being aniiwcfcd 
correctly without reading the paeBage have boen Included with 
increasing frc-qucncy in revlsiona of thcs widely uBcd Btand«rdlged 
reading tcstg. The paaeage forni the Met c ropo 1 1 1 on cited above can be 
used to illustrate this type of qucatlon. The end of that passages 
contains tho following four sentences^ 

"But last year, some poople came in huga truck ond chopped down 
all the oak trees. They wontod to make the atrect wider. 1 hope 
the equirrelg and the birds will return to our block. The other 
children want then back too, ao today wo planted six soall 
trees." 

One of the quefltlona following this passage lai 

"After the trees were cut downj the children aissed the - 

A. trucks C. men 

B, anioals D. nolae" 

Hithout the passage, all of the optlone are plausible, i.e., It is 
reasonable to expect taat the queatloo is passage dependent. The 
question also requires an integration of the concept of "animals" with 
"the squirrels and the birds" and an inference that "I hope" and the 
"other children want" because the squirrels and birds were "missed". 
On the other hand, the child who depends heavily on prior koowledge 
and personal preference may reasonably Infer that the children missed 
the trueks more than the animals. As noted by Valencia and Pearson 
(1986), Individual differences in prior knowledge can lead to 
differences in the inferences that are made from a given text, and 
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«or«! thflu one Inference Is ofttjn def ena 1 bi e . 

The DegreeB of Reading Power CTouchstofie AppXlcd Sclonccj 
ABBOclfltcB, 1983) llluBtrateB another approach to teattng tho 
integration of Information acroos sentences. The DRP has features 
similar to a cIoec test. Words are deleted from « passage and the 
test taker has to tele one of five optlona that should be used for 
the deleted word. It differ, froo a typical cIoeo test, however. In 
several important re.pects. First, fewer words are deleted (only 
aeven per passage which typically run approxlao to 1 y 325 words In 
length) than on the uaual cloie teat. Second, the deleted word and 
wordo in the sentence where it belongs are all famUlar, relatively 
high frequency worda. That Is, lo cofflparlson to the aurroundlng 
.entencea. the target word and sentence 1. easy. Thus, the dependence 
of the DRP scoroa on the particular vocabulary of the reBponse optlona 
i. minimized. Third, if only the carget eentence la considered, all 
of the optiona are reasonable. This is an esaentlal part of the DRP 
approach. It is Intended to ensure "that procesBing surrounding prose 
la both necessary and sufficient to choosing the right answer" 
(Koslln, Koslln, k Zeno, 1979, p. 316). 

The following .ample iten from Form PX-1 of the DRP illustrates 
this key third feature. 

"It was sunny and hot for days. S-i a) price b) road 

Then the s-l changed. el <r,K j^ .u 

* Job d) weather 

It turned cloudy and cool. size" 
Although shorter than the aetual test paBsages, it can be seen that 
there is no good baala for choosing among the four optional words when 
the target sentence is considered in isolation. When integrated with 
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Che Information in thci preceedlng and flubfler4ucnt ocntenccii, however, 
only 'VcttCher" leads to ii coherent set. 

The examploa from the Metropolton ond the DRP uhov that existing 
test Iterae can and do tap aomc of the integrfltlve aepcctB of sentence 
proceSBlng that were highlighted by Curtle and Glaaer (1983), Other 
aspecta, however, geom to be mlBsed, For example, Blnce the pflflsage 
1* available for reference when the quoBtlons are considered, it lo 
not neceBBary to rely «s heavily on the retrlval of concepts from 
long-tera metnory as way be necessary In flkllled reading. Fewer 
demands may also be made on the oanlpulatlon of concepts in ihort-terra 
nemory. Certainly, a teat on which the text was unavailable would 
change the task demands. Whether such a procedure would provide 
additional useful information about individual differences In reading 
ability Ig an open question, however. 

Integrative procesBlng servea as a baals for detecting and 
recovering from misinterpretations of text. Markman (1977, 1979) has 
damonstrated that children who have difficulty In understanding text, 
even text containing only slBple words that the chi on can readily 
decode and that are familiar in moaning, are frequently unable to 
detect Inconsistencies. This Is illustrated by one of his examples; 
"Fish must have light In order to see. There la absolutely no light 
at the bottOB of the ocean, ... Some fish who live there know their 
food by its color" (Markman, 1979, p. 646). Failure to detect the 
IneonBltencey is symptomatic of the child's lack of integration of the 
Information contained In the third sentence with the information 
provided earlier by the first sentence. The use of inconsistencies 
such as the one illustrated above represents a promising approach to 
the construction of reading tests that assess a child's ability to 



28 



Integrate 1 nf m cj t Ion ucraBW dlffereru parts of a text. 
Pa BBa^c C Q a p r c h c n s i on 

According to Anderson (1977, p. U9) "Itjo comprehond a mu«..g. 
IB to dl8cov«r n formulation which cohorontiy e^pl«ln« itu contcnta." 
Coopreheuslon of the BOBsage In a text roqulre. more th«n the ability 
to decode and attach meanings to words. It requiroB more than the 
ability to recognize paraphrases of the sentencoa. It Involvea tha 
••forBlng of a coherent cognitive modol of the text Beaning" (Johneton, 
1984, p. 236). Development of an appropriate cognitive nod 1 dopends 
on the schcoa that the reader poseeases as well as the properties of 
the text (e.g., Anderson, Reynolds, Schflllert & Goctz, 1977; Branaford 
i Johnson, 1973; Rumelhart & Ortony, 1977). 

Schema theory attempts to explain how knowledge is stored In 
memory; how It Is recalled, used and elaborated; and how It is uatd in 
comprehension. ConBlder.ble progress has been made In the elaboration 
ot schema theory In the past decade (see, for example, Anderson & 
Pearson, 1984; Brewer & Nakamura, 1984), It has played a major role 
In the direction of research on reading and is beginning to have an 
Impact on the design of instructional materials. To date, however, 
there seems to be little, if any Indication that schema theory has had 
an Influence on the nature of standardlEed reading tests. 

Schmata can involve both declarative and procedural knowled . and 
can have a .^rong influence on an individual's ability to comprehend 
and remember text. The role of schemata In facilitating comprehension 
was nicely Illustrated by Curtis and Glaser (1983) using the following 
excerpt taken from Brangford and Johnson (1973). 

"The procedure is actually quite simple. First you arrange 
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things Into different groupo. Of couriso, one pile raoy be 
Bulflclenc depending on how much there Is to do. If your have to 
go Boniewher© elac due to lack of focllitlco that In the next 
step, otherwise you are pretty well set. It La ioportanc not to 
overdo things. That Is, It la bettor to do too lew things at 
once than too many. In the short run this may not seem important 
but conpllcatlons con uoslly arise" (Brfinsford & Johnson, 1973, 
p, 400). 

Tho words and sentences ore straight forward, but It Is diflleult to 
construct a coherent Interpretation of the passage. However, If told 
that the passage is about washing clothes, the deBcrlptlon fits an 
already available schoata and Is auch nore readily underatood. 

PoBseselon of appropriate achftuto facilitates comprehension. 
This concluaion has useful Implications for Instruction, but poses a 
dlleuma for test construction. Since individuals differ In the 
schemata, the prior knowledge that they bring to the test aUuatlon, 
the choice of passages for the test can obviously be an loportant 
determinant of the-relatlve performance of different indlvlduali. 

Results of a study by Johnston (1981) demonstrate that 
differences In the match of test passages to the prior knowledge of 
test takers can have « substantial effect on performance. Johnston 
constructed a test based on two passagess onn dealing with the 
speclaiization of corn in the United States and one dealing with the 
financial problems of the Chicago Regional Transit Authority (RTA). 
Both passages were administered to a sample of students In a rural 
community and to a sample from an urban community. As predicted, the 
rural sample outperformed the urban sample on the test based on the 
corn passage, whereas the converse was true for the RTA passage. 
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AHhough not explicitly au n confl^quenca of schema theory, toat 
constructors have «ttempt«d to avoid, or more accurately, to nlnlmlz« 
the effccta of prior knowledge on reading test per f ortnance. Topics 
asaumed to require spaclallaed prior knowledge are avoided. Tha 
prlnary way in which the effect, of prior knowledge are ainlmlEed. 
howuver, Is by the use of multiple pflssagoB .hlch are flelected in waya 
that attempt to balance the relative advantagei and dlaad vantagea of 
different groups of test takers. One of the conBoquencea of this 
approach, however, Is that teats typically consist of a number of 
short passages on different, sometimes obicure. topics that hold 
little motivation or interest for the reader. Furthernore, this 
approach does more to conceal than to eliminate the effects of prior 
knowledge (Valencia 4 Pearson, 1986). 

The reading test of the EleBentary level of the Motropolltan 
Achievement Tests, Form JS, for example, contains U passages ranging 
from 89 to 148 words In length. The passages deal with such varied 
topics as earthworms, bakeries, maple syrup, a turtle race, and a 
fanciful story about a wood chopper. 

Short passages that are typical of reading tests place fewer 
cognitive demands for the formation of an Intergrated representation 
of the information than do longer segmerts found in textbooks and 
other reading materials that students are expected to read and 
understand. This suggests that longer text segments may need to be 
used. Of course, feasibility would dictate that if longer passages 
passages were used, they would have to be fewer in number. As a 
consequence, such tests might be less fair than existing tests because 
they would lack the same degree of balance across a number of topics 



that iB mode poselblo by the use of nany ahort poasngcB. 

Nonetheless, attong argutDeota in favor of longer pflssagos can be 
made. Longer pa0Bage8 can be made more comparble to the laportant 
reading taskB that studentB are asked to perform In school. They can 
involve more meaningful and coherent BtorleB or pleeea of espo»itory 
text. Thus, some reading reeearcheri (e.g., Pearson, personal 
comiDunlcatlon) have argued that alternative approaches to the 
potential problems of bias that may be introduced by the uae of a 
small number of longer pasBages should be sought. 

The solution to the dlleaaa may depend on the creation of 
techniques for assessing relevant prior knowledge prior to 
administering a reading test. This was attempted with some succeos by 
Johnston (1981) using passage specific vocabulary test Items. Several 
other posslbllitleB have been suggeated, some of which are currently 
under Investigation. For example, Curtis and Glaser (1983) have 
suggested that comparisons of performance In reading and listening 
comprehension as another possible approach to this problem. 

Pearson and Valencia are currently conducting research for the 
Illinois Assessment Program in which several approaches to the 
assessment to prior knowledge are being investigated. Using a variety 
of reapoose formats, they ask children to make predictions about what 
they are likely to find in a passage on a given topic or to indicate 
which of a number of coooepts are apt to be closely or somewhat 
related to the topic of the passage. While it la too early to know 
which techniques will prove most effective, this work Is Important to 
the development of reading tests that involve longer passages with 
greater ecological validity that also take into account Individual 
differences in prior knowledge. 



Whotuver the nature ot the paflsflges, meaouroiiient of possage 
comprehension genorflHy depends on quesUona that requlru the test 
taker to make Inferences. PubllBhed reading teats uso a variety of 
approaches In attempting to Bcasure a test taker's ability to draw 
inferences from text. ExaBpleB include itCBi that ask the test taker 
to select the best title for a atory, to indicate the author'a intent 
or point of view, or to make Inferences about the feelings of a 
character or likely explanations of events. 

The following 1 1 lustrations of questions that are Intended to 
assess the ability of children to draw Inferences from a passage were 
taken from Level E of the CTBS, Form U. 

••Which of these would most likely be a Norman Rockwell picture?'* 

"Why was this animal called Thunder Lizard?" 

"What is this story laostly about?" 
Some similar examples from the Advanced I Level of the Metropolitan, 
Form JS are: 

"The best title for this story Is - " 

"The author feels that ctty play streets are 

"At the end of this story, the actions of the lovers were 

motivated by their feeling of 
It would appear that none of the above Illustrative questions can be 
•nswered by simply matching surface features of the question and the 
passage. They all require some degree of inference to be made based 
on the test taker's comprehension of the passage. However, answers 
can soaetimes be determined by simple asaociatlons and the types of 
Inferences required are fairly limited. Moreover, because of the 
nature of the short test passages and because the text is available 



for rcfefGnce In answering the queBtlonip such questions do not 
cxhauot the rang© of Integrative and inforentlal flklllB required In 
many real-world reading tasks. 

As noted in the previoUB seetion, for exarop 1 o , an Integrated 
repreaentfltlon of a text enables a reader to Identify inconalstencles 
in text. Markroan (1 979) argued for the Importance of 1 neons i g tency 
detection as followai "to notice inconalB tencles children hav^ to 
encode and store the information> draw the relevant lnferences» 
retrieve and aalntaln the (inferred) propositions In working ffieiaoryp 
and compare then" (p. 6^3), Though seldom used on reading tests, an 
approach that requires a test taker to identify incons Jo tenclee In a 
text may provide a powerful means of assessing the degree to which 
Information from different parts of text is integrated Into a coherent 
reprasantatlon of the meisage* Care would obviously need to be taken, 
however, that laconsls t eneles are crucial to understanding Important 
aspecta of the text« 

The fact that a test taker can refer back to the text whan 
raspondlng to test questions, aspeclally tax t^exp 1 Icl t ones, reduces 
the demand to develop an Integrated representation of the passage* 
This suggests that it may be desirable "to prevent the reader from 
rafarring to the taxt while answarlng the quastlons" (Johnston, 1984, 
p. 236). 

Anothar alternative approach to tasting that has racalved 
incraasing attention in the past few years Is typlfiad by the work of 
Camplona and Brown (1985) on dynamic assessmant* The emphases In 
dynamic assassmant are on change that occurs during ralaclvely brief 
periods of guided instruction and on the ability to tranefar what is 
learned to new situations^ A series of studies revlewad by Camplona 
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and Brown (1985), indicate that tho best predictors of future learning 
are not stotlc testa, but the chongea that take place during tho 
dynaBlc aeseaamont process and BOflBurcB of a student's ability to 
"tranefer their newly learned skills to novel Bltuatlons" (p. 35). 
Although the typoa of taaka (e.g.. Ravan Pgrogreaalva Hatrlcea tasks) 
that hove been most used In research on dynamic aBsessoant are quite 
different from those needed to asBess reading conprehension . the 
approach is certainly worthy of careful consideration. 
ConcluBion 

There Is a relatively poor aatch between current theory and 
experl«ental research on the reading process and cxieting atandardited 
testa of reading. This lack of correspondence may be due, In part, to 
a difference in goals and, in part, to a lag between research and 
practice. Survey testa have long followed a traditional psychooeeric 
perapectlve derived from the goals of ranking and sorting students 
baaed on Individual differences In performance. Normative comparisons 
and predletive validity have been the hallfflarka of this approach. 
WlthlB this tradition, current standardleed tests are relatively 
effective and quite efficient. The diatlnctlons among the 
Interrelated componenta of reading articulated by Curtis and Glaser 
(1983) which have provided the framework for this review, are of 
little Importance to the traditional goals of norm-referenced tests. 

Diagnostic tests and criterion-referenced tests have followed a 
different tradition. Tfet the aatch between these tests and the 
theoretlca rspecelve emphasized In this report Is equally poor. 
Such tear generally coneelve of reading as a host of hierarchically 
organize .acrete skllli. Little emphasis Is given to the 
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Integrative! proccssei Involved in reading. 

It remains to be seen whether tests derived from modern 
cognitive theories of the reading proccias will lead to subBtantlal 
improvements in reading assessment. Certalnlyi it Beams unlikely that 
substantial improvements In prodlctlon will result# Howeveri 
prediction is not a primary goal for such tests* Providing more 
inat ruct lona 1 ly relevant information Is* As Curtis and Glasar <1983p 
p* 144) concluded, '"[a] comblnad enterprlae representing teat design 
based on knowladge of human learning and performance, piychoaetrlc 
requirements I and studies of test use should improve our ability to 
link testing and inii t ruct 1 on*" 
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